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Abstract 

In the wake of recent school shootings, fear over violence in schools has prompted increased 
requests for psychologists, educators, and law enforcement professionals to assist in preventing 
future school violence incidents. We attempt to lay a foundation for developing effective 
assessment and prevention approaches by first distinguishing planned school-based attacks from 
other forms of school and youth violence. We then review the three assessment approaches that 
have been advocated and used in some jurisdictions (profiling; guided professional judgment; 
automated decision-making) and demonstrate why they are inappropriate— and potentially 
harmful— for preventing planned school-based attacks. We then describe the contours of the 
threat assessment approach, developed by the U S. Secret Service to prevent assassinations, and 
examine its utility for responding to communications or behaviors of concern that students may 
present in school settings. 
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Introduction 

The recent school shootings that have occurred in the past few years in several 
communities across the country, including Pearl, MS, West Paducah, KY, Jonesboro, AR, and 
Jefferson County, CO, have raised safety concerns and fears among students, parents, and school 
administrators nationwide. These rare, but highly salient incidents, such as the one at Columbine 
High School, have garnered considerable attention from the national media (Arnette & 

Walsleben, 1998; Brooks et al., 2000; Elliott et al., 1998; Lawrence, 2000), and appear largely to 
be responsible for the surge of public concern (Stossel, 1999). In the wake of these infrequent 
but highly-publicized events, school administrators, mental health professionals, law 
enforcement professionals, and policymakers have come under increasing pressure to take steps 
to prevent school shootings in their communities (Brooks, Schiraldi, & Ziedenberg, 2000; 
Lawrence, 2000; Sugai, Sprague, Homer, & Walker, 2000). In their quest to avoid becoming the 
next statistic or headline, those with responsibilities for preventing school shootings have 
focused preventive resources primarily on increasing physical security (e.g., installing cameras 
and metal detectors), hiring school security officers, developing tactical plans for responding 
once a shooting has occurred, and implementing a range of programs such as legal education and 

4 

conflict resolution. Unfortunately, these responses are not likely to be effective in preventing 
planned school-based attacks. 

In this paper, we focus specifically on approaches for preventing planned school-based 
attacks, rather than on other more common and recurring forms of school violence. We 
conceptualize and refer to such incidents as examples of “targeted violence” - violent incidents 
where.both the perpetrator and target(s) are identified or identifiable prior to the incident 
(Borum, Fein, Vossekuil, & Berglund, 1999; Fein & Vossekuil, 1998; Fein, Vossekuil, & 
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Holden, 1995). The defining element of targeted violence is that the perpetrator selects a target 
prior to the violent incident. In some instances of targeted violence the target may not become a 
victim; for example, the target may avoid injury (e.g., if the perpetrator is a poor shot) or the 
target may not be at the site where the perpetrator believes the target to be. In others the target 
may be one of several victims or may be the only victim. In still others, the target may be an 
institution or facility, as it was in Timothy McVeigh’s bombing of the Murrah Federal Building 
in Oklahoma City. But each case would be an instance of targeted violence if a target was 
known-or was knowable-prior to the incident (Borum et al., 1999; Fein & Vossekuil, 1998; Fein 
etal., 1995).' 

In the wake of recent school shootings, a cottage industry of school safety products has 
quickly developed (Chaddock, 2000). To date, however, schools have been given little or no 
empirically-based guidance on how best to assess the risk posed by a student for targeted 
violence in schools. 2 In our view, the current options available to schools for responding to and 
preventing school shootings— which we review herein— are at best unproven (Hoagwood, 2000) 
and at worst carry the potential for serious harm (Hyman & Perone, 1998). Some violence 
prevention options, such as zero tolerance policies, have been criticized as overly punitive 
(Hyman & Perone, 1998; Tebo, 2000). Others, such as behavioral profiles and computer 
programs that identify students at risk, have raised widespread concern among parents, students, 
policymakers, and even the U S. Secretary of Education for their potential to infringe on 
students’ civil liberties and to unfairly label or stigmatize certain students as “dangerous” 
(Cooper, 2000; Morse, 2000; Sewell & Mendelsohn, 2000; Steinberg, 2000). 

In this paper, we attempt to lay a foundation for developing an effective assessment 
approach to evaluate the risk of targeted violence in schools by addressing four issues. First, we 
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delineate the contours of the problem of targeted violence by distinguishing the fear of this 
violence from its actual probability and by distinguishing targeted violence from other forms of 
aggression in youth. Second, we examine and critique three assessment approaches-profiling, 
guided professional judgment, and automated decision-making-that have been advocated and 
used in some schools to identify students at risk for violence, giving particular consideration to 
the potential for harm inherent in these current approaches. Finally, we describe and explore the 
utility of a threat assessment approach (Borum et a!., 1999; Fein & Vossekuil, 1998; Fein et al., 
1995) to identify and assess risk posed by a potential student perpetrator for targeted violence in 
school. 

Delineating the Problem 

The problem of “school violence” has been defined and constructed in a myriad of ways 

(Furlong & Morrison, 2000). While eliminating all forms of antisocial aggression and violence 

in youth is a laudable goal, different types of violence have different antecedents and thus require 

different approaches for assessment and intervention (Cornell, 1990; Dishion, McCord, & 

Poulin, 1999; Hoagwood, 2000; Quay, 1987). The first step in developing effective assessment 

approaches and appropriate policy is to identify clearly the types of behavior or outcomes that 

# 
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one is trying to prevent (Chavez, 1999; Furlong & Morrison, 2000). 

Fear of school violence vs actual probability . It is clear that many Americans fear 
violence in schools, but the precise nature of this fear and its relationship to the actual probability 
of harm are much less certain (Arnette & Walsleben, 1998; Elliott et al., 1998; Furlong & 
Morrison, 1994; Henry, 2000; Hyman & Perone, 1998; Hyman et al., 1997; Lawrence, 2000; 

US. Department of Education & US. Department of Justice, 1999). Many have argued that the 
extensive media coverage of incidents of targeted school violence, and its disproportion to the 
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actual prevalence of targeted school violence, have significantly exacerbated fear over school 
violence (Amette & Walsleben, 1998; Brooks et al., 2000; Elliott et al., 1998; Henry, 2000; 
Lawrence, 2000; Snyder & Sickmund, 1999; Stossel, 1999; U.S. Department of Education & 

U S. Department of Justice, 1999) 4 People seem to fear school-based homicides most; yet, 
statistically these events are so rare that the epidemic of concern would seem misplaced. 

Relative to the risk of violent victimization that children face outside of school, the risk 
they face in school is minimal (Goldstein & Conoley, 1997; Hyman et al., 1997; Kaufman et al., 
1999; U.S. Department of Education & U.S. Department of Justice, 1999). The most recent 
report on safety in American schools indicates that while more than 2500 children in the U.S. 
were murdered or committed suicide in the first half of the 1997-1998 school year, less than one 
percent of those deaths - including those from multiple victim homicides — occurred at school 
(U.S. Department of Education & U.S. Department of Justice, 1999). 5 While the odds that a 
student would be threatened or injured with a weapon at school during the 1996 year were 
approximately one in 15 and the odds of getting into a physical fight at school were 
approximately one in eight (Snyder & Sickmund, 1999), recent studies have estimated that 
during the past three academic years the odds that a child would die in school (by homicide or 
suicide) were no greater than 1 in 1 million (tj.S. Department of Education & U.S. Department 
of Justice, 1999) - and some argue were closer to 1 in 2 million (Brooks et al., 2000). 

Although these incidents are extremely rare, they are so vexing and their impact is so 
great that the fear they engender can often drive radical policy change, in some cases leading to 
the implementation of bad policy (Hyman & Perone, 1998). Accordingly, we view the fear of 
school violence as a distinct problem that schools face, and one that must be acknowledged and 
considered in developing and implementing prevention policy. 
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Targeted violence vs. general aggression . As noted above, different types of violence 
have different antecedents and thus require different approaches for assessment and intervention. 
Identifying children and adolescents who are at risk for violent behavior, broadly conceived, is 
not particularly difficult. There is an extensive empirical knowledge base of risk factors 
(Hawkins et al., 1993, 2000); and, violent behavior during adolescence is so common that, in 
some groups, it is virtually normative (Elliott, Ageton, Huizinga, Knowles, & Cantor, 1983, 
Hirschi, 1969; Moffitt, Lynam, & Silva, 1994). Events that occur so frequently are, statistically, 
not as difficult to predict. 

The particular challenge that schools face in trying to prevent targeted violence in school 
is to assess the nature and degree of risk posed by a student who has come to official attention 
because of some threatening communication or behavior of concern. The question is not 
whether the student might be at increased risk for engaging in some form of aggressive behavior 
during adolescence, but rather whether he or she currently poses a substantial risk of harm to 
another identified or identifiable person(s) at school. Students at risk for targeted violence may 
or may not possess many of the traditional risk factors associated with general violence 
recidivism and delinquency in youth. The etiology and intervention for targeted violence may 

i 

differ substantially from more general forms of aggressive behavior in youth. Indeed, studies of 
juvenile homicide suggest that youth who commit murder differ along certain dimensions from 
those who engage in nonviolent delinquency (Cornell, 1990; Cornell, Benedek, & Benedek, 
1987a, 1987b)-but in ways that may seem counterintuitive. For example, Cornell and his 
colleagues found that, compared with juveniles who were referred for evaluation after 
committing larceny, juveniles who were referred for evaluation after committing homicide were 
less likely to have prior mental histories, less likely to have a history of prior arrests or placement 
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in a juvenile facility, and less likely to have had problems with school adjustment. Youth 
convicted of homicide were less likely to have histories of prior violent behavior than were 
juveniles convicted on assault charges. (Cornell, 1990; Cornell et al., 1987a, 1987b). Notably, 
there is considerable heterogeneity even among juvenile homicide offenders (Cornell, 1990; 
Cornell et al., 1987a, 1987b). Youth who commit acts of targeted school violence may differ 
substantially not only from juveniles who engage in nonviolent delinquency but also from other 
juveniles who engage in different types of homicide. 

Current Approaches to Targeted Violence Assessment 
In the sections that follow, we examine the three assessment approaches currently 
advocated and used in some jurisdictions 6 for evaluating risk of targeted violence in schools. 
These are: (a) profiling; (b) guided professional judgment / structured clinical assessment (which 
includes the use of warning signs and other checklists); and, (c) automated decision making 
(which includes the use of actuarial formulas and expert systems). For each of these approaches, 
we describe how the evaluation is conducted; identify its threshold for concern used to determine 
risk of targeted violence in school; explore its utility for evaluating risk of targeted violence in 
schools; and, examine the potential for harm (to students and others) inherent in each. We then 
turn to the threat assessment approach, the strategy developed by the U.S. Secret Service for 
identifying, evaluating, and managing threats and other inappropriate behaviors directed toward 
the President and other public officials (Borum et al., 1999; Fein & Vossekuil, 1998; Fein et al., 
1995). After describing the contours of the threat assessment approach, we explore its utility for 
school administrators, law enforcement professionals, mental health professionals, and others to 
determine the risk of targeted school violence posed by a student who has engaged in threatening 
or otherwise concerning behavior. 
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Profiling 

The term “profiling” has become increasingly familiar in recent years. It is used broadly 
to connote a range of identification techniques or assessment strategies that are used in both law 
enforcement and non-law enforcement settings (Hormant & Kennedy, 1998; Turvey, 1999a). 
Recently, the practice of profiling has come under considerable scrutiny as concern has 
developed both over the use of racial profiles to single out types of people pulled over in traffic 
stops (e g., Rogers, 2000), and over the use of demographic or behavioral profiles to identify 
types of students likely to become “school shooters” (Cooper, 2000; Morse, 2000). To clarify 
what is meant by profiling and establish a foundation for better understanding recent concern 
over its use in schools, we will review briefly the various techniques that share the term 
“profiling” and then describe in greater detail the profiling techniques suggested by some to 
determine risk of targeted violence in schools. 

Description and threshold of concern. The technique of crime scene profiling, as 
originally developed by the Federal Bureau of Investigation’s (FBI) Behavioral Science Unit, 
involves using information gathered from a crime scene to generate a set of hypotheses about the 
characteristics— physical, demographic, personality, and others— of the person most likely to have 

t 

committed the crime (Douglas, Ressler, Burgess, & Hoffman, 1986; Holmes & Holmes, 1996; 
Homant & Kennedy, 1998). The hypotheses are then used generally to help narrow a list of 
suspects or to suggest other areas of investigative inquiry and thereby enhance the efficiency of 
an investigation. This technique is retrospective in that it works from a behavi or (i.e., the crime 
and crime scene evidence) backward to infer the type of person who committed the crime. This 
technique has gained popularity among state and local law enforcement personnel, with 
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anecdotal reports of considerable success (Douglas et al., 1986) and some limited empirical 
support (Homant & Kennedy, 1998; Pinizzotto & Finkel, 1990; Turvey, 1999b). 

The technique of criminal profiling has since been extended to include techniques for 
prospective identification of would-be criminals. Rather than starting with a crime and working 
backward to the type of person who committed the crime, the prospective form of profiling 
begins with a specific person (one suspected of being, for example, the next “school shooter”) 
and projects forward to try to predict the future likelihood that the person in question will 
commit the crime of concern. In this version of profiling, a profile or description of the typical 
perpetrator of a particular type of crime-such as serial murder or school shootings-is compiled 
from characteristics shared by known previous perpetrators (Homant & Kennedy, 1998; 
Pinizzotto, 1984). This prospective profile is then used as a prototype or template against which 
an individual who is suspected of being (or of becoming) a perpetrator may be compared. 
Prospective profiling is used both to identify types of individuals likely to become perpetrators 
(absent a behavior or communication that brings someone to official attention) and to assess a 
given individual who has come to someone’s attention for some troubling communication or 
behavior. The threshold for concern in both cases is a sufficient degree of “fit” 7 or similarity 
between the characteristics of prior perpetrators and those of the person under consideration. 
Various agencies and professionals have developed prospective profiles of “the school shooter,” 
including the school shooter profile developed by the FBI (Band & Harpold, 1999) and the 
“classroom avenger” profile developed by McGee and DeBernardo (1999). 8 

Attendant problems . Numerous concerns have arisen over the use of prospective 
profiling to identify and assess the risk students pose for targeted violence in school. First, 

i 

prospective profiling is neither sufficiently sensitive nor specific to identify a child who may be 




11 



School Threat Assessment 1 1 



at risk for engaging in targeted school violence, nor for evaluating the child’s likelihood of doing 
so (Sewell & Mendelsohn, 2000). Prospective profiling to identify students likely to become 
“school shooters” carries with it considerable risk of false positives; that is, because targeted 
violence in school is such a rare event, most who “fit” the profile will not engage in acts of 
targeted school violence (Sewell & Mendelsohn, 2000). In addition, use of prospective profiles 
would inappropriately exclude students who do not fit the profile, but who may in fact pose a 
risk of targeted violence. By way of example, the use of a prospective profile derived from 
previous assassins would have failed to identify Sarah Jane Moore prior to her assassination 
attempt on President Ford in San Francisco in 1975. The profile most accepted at that time 
would have predicted Ford’s attacker to be male, between the ages of 20 and 40, of slight build, 
bom overseas, unemployed, a loner, and someone who suffered from delusions of grandeur or 
persecution (Weisz & Taylor, 1969). At the time she shot at Ford, Moore was female, in her mid 
40s, of stocky build, bom in the U.S., employed full time as an accountant, had been married and 
had a son, and had no history of delusions. 

Second, the accuracy of school shooter profiles is questionable. The FBI’s offender 
profile is based on only six school shootings (Band & Harpold, 1999; p. 14), whereas other 
research has identified over 40 cases of school shootings in the past 20 years (Henry, 2000, 
Vossekuil, Reddy, Fein, Borum, & Modzeleski, 2000). There is no discussion in the FBI’s 
profile regarding the extent to which the information taken from the six shootings is 
representative of the over 30 school shootings not included in the profile. Given that incidents of 
targeted violence in school are so infrequent, we would caution against any generalizations being 
made from such a minority of cases. In addition, the classroom avenger profile (McGee & 
DeBemardo, 1999) erroneously describes all of the perpetrators they included as white males. 
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when in fact three of them were not (Sincino was African-American, Sirola was Hispanic, and 
Ramsey is half Native Alaskan). This kind of inaccuracy calls into question the accuracy of the 
other characteristics presented. It also highlights the fact that information about the perpetrators 
and offenses is only as accurate as the source from which it is derived. Verlinden, Hersen, and 
Thomas (2000a, 2000b), for example, relied exclusively on media accounts of school shootings; 
yet, preliminary findings from the first empirical study of incidents of targeted school violence 
suggest that, when compared with investigative and court records, media depictions of school 
shootings are in many cases incomplete or even inaccurate (Henry, 2000; Vossekuil et al., 2000). 

Third, there are no data that demonstrate the validity or effectiveness of prospective 
profiling to identify potential perpetrators for any type of crime. What little empirical support 
exists for the use of profiling exists for retrospective identification only (Grubin, 1995; Homant 
& Kennedy, 1998). 9 Moreover, research on decision making suggests that the use of prototypes 
to determine whether someone or something belongs in a particular category (for example, 
whether a student may be a “future school shooter”) can increase the effects of judgment bias 
Decision makers who rely on characteristics that appear to be more typical or representative of 
the category, to determine whether an object belongs in that category, may inadvertently render 
faulty decisions (Kahneman & Tversky, 1972* 1973; Nisbett & Ross, 1980; Sewell & 
Mendelsohn, 2000; Tversky & Kahneman, 1974). For example, a school administrator who 
believes she should be concerned about a particular student because the student wears a black 
trench coat similar to the ones worn by the shooters at Columbine High School would be relying 
inappropriately on such information to determine risk. This use of what is called the 
representativeness heuristic can bias decision making when the characteristics an evaluator uses 
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(e g., “wears black trench coat”) are in fact less informative than others that may appear less 
typical of previous shooters (e g. , has asked friends where he could get a gun). 

Still other research suggests that when decision makers set out to determine the validity 
of a working hypothesis (e g., that a particular student fits the school shooter profile), they may 
inadvertently search only for information that confirms their hypothesis and fail to search for - 
or tend to discount - any information inconsistent with the hypothesis (Borum, Otto, Golding, 
1993; Nisbett & Ross, 1980; Snyder & Swann, 1978). Known as the hypothesis confirmation 
bias, this effect may increase the risk of false positives if the evaluator begins with the 
hypothesis that the student in question fits the profile - rather than beginning with the hypothesis 
that the student does not fit the profile (Borum et al., 1993; Sewell & Mendelsohn, 2000). 

Finally, use of prospective profiling in schools has received extensive criticism from 
those the approach is designed to benefit, including parents, students, and even the Secretary of 
Education (Cooper, 2000; Morse, 2000). This criticism has focused primarily on the risk of 
unfairly labeling students as dangerous and the potential for stigmatizing them and depriving 
them of their civil liberties as a result (Sewell & Mendelsohn, 2000). Criticism has also focused 
on the potential for profiling to produce bias, particularly bias against students who differ from 
the majority in terms of appearance, race, sexual preference, etc. Results from a 2000 survey 
conducted by Time magazine and the Discovery Channel indicate that the majority of students 
polled (60%) disapprove of the use of profiling in schools (Morse, 2000). Their concerns and 
fears are based on the potential for unfair use of profiling against students who are not likely to 
be violent. Secretary of Education Richard W. Riley publicly opposed the use of profiling in 
schools to identify potentially violent students, saying that we “simply cannot put student 
behaviors into a formula to come up with the appropriate response” (Cooper, 2000; p. All). 
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Guided Professional Judgment 

The second approach used in schools to evaluate the risk of violence posed by a student is 
that of guided professional judgment, also referred to as structured clinical assessment. In 
general, clinical assessments that form the basis for professional risk judgments involve an 
interview and evaluation of an individual that is informed by the base rates for violence within 
the individual’s population and by relevant risk factors known to be related to the risk of violent 
behavior (Borum, 1996, in press; Otto, in press). When the evaluator uses instruments or 
checklists that help to structure or guide the collection and analysis of appropriate information, 
the approach is referred to as guided professional judgment or structured clinical assessment 
(Borum, in press; Otto, in press). We use the former term here since those conducting these 
assessments in schools may not necessarily be mental health professionals. Risk judgements 
based on guided or structured assessments have been shown to produce greater rates of accuracy 
than those based on unstructured assessments for civil and forensic psychiatric patients, sex 
offenders, and domestic violence offenders (Dempster, 1998; Hanson, 1998; Kropp, Hart, 
Webster, & Eaves, 1999). 

Guided professional judgments are traditionally conducted by trained and licensed mental 
health professionals (Borum, 1996; in press)* ' However, school officials and law enforcement 
personnel also use checklists of risk factors and warning signs-such as those put out by the 
Departments of Justice and Education (Dwyer, Osher, & Warger, 1998), 10 the International 
Association of Chiefs of Police (1999), and the American Psychological Association 
(APA)/MTV collaboration (APA, 1999)— in a less formal but similar manner to that used in 
structured clinical assessment. Therefore, we have broadened our discussion to include the use 
of checklists and warning signs by school and law enforcement personnel. Many of the concerns 
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about the use of guided professional judgment to evaluate risk of targeted school violence apply 
to school and law enforcement use of checklists and warning signs as well . 1 1 

Description and threshold of concern . In guided professional judgment, to the extent that 

the assessment requires an estimate of probability or relative risk, the evaluator may begin by 

determining the base rate for the type of violence in question, among individuals with similar 

demographic or clinical characteristics. This provides a baseline estimate of the probability of 

violence among people in relevant populations. The evaluator then gathers information from and 

about the individual by consulting a checklist of factors, each of which has a demonstrated 

relationship to violence recidivism based on existing professional literatures, and each of which 

may have some form of scoring criteria (Borum, in press). This approach helps the evaluator to 

gather all relevant data during the course of interviews with the individual and reviews of 

existing records (e.g., school, mental health, etc.). The evaluator may then adjust (or not) the 

baseline probability of risk up or down, depending on the presence or absence of relevant risk 

factors, to determine the individual’s risk for violent behavior. 12 The goal of a guided 

professional judgment is to reach an appraisal of risk that is well informed by the best available 

research (Borum, in press). The threshold of concern is the presence, within the individual or his 

* 

or her situation, of a sufficient number (or severity) of risk factors known to be related to 
increased risk for violent behavior. 

Attendant problems . We recognize that guided professional judgment represents a best 
practice approach in the mental health professions for assessing risk of general aggression, in 
school as well as in other settings. However, we believe there are several limitations to its use 
for determining the risk that a student poses for targeted violence in school. First, because the 
prevalence of targeted school violence is so small, an evaluation of risk for targeted school 
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violence cannot be driven primarily by the base rate (Sewell & Mendelsohn, 2000). 13 If it were, 
the evaluator would have to start with such a low probability baseline that even if all known risk 
factors were present in a given student, the final appraisal of violence likelihood would 
necessarily be low. For example, if a student was known to have recently acquired a weapon, 
and had told several friends that they should not go near the library at 9.00 a m. on Friday, a 
school administrator could not reasonably ignore this information just because the base rate for 
school shooting is miniscule. Sewell and Mendelsohn (2000) have argued, and we agree, that the 
cost of making a Type I error in such a case-that is, of assessing the student as not posing a risk 
when in reality he or she does— would be too high. 

Second, empirical research is not yet available on risk factors for targeted school 

violence, so that evaluators do not yet know what information to gather for a school-based, 

targeted violence risk assessment (Borum, in press). As we noted earlier, different types of 

violence have different antecedents. Thus, we believe it will be necessary to conduct empirical 

research on the antecedents of targeted school violence before a sufficient knowledge base can 

inform guided professional judgments for this type of violence. Third, and relatedly, existing 

empirical research on general violence may have limited utility for assessments of risk for 

1 f 

targeted school violence (Borum, in press; Borum et al., 1999). It is unclear how aggregate data 
from research studies on other types of youth-perpetrated violence will generalize to specific 
targeted violence fact patterns (Borum, in press). Most of this research has examined only 
general violence recidivism as a criterion. Moreover, most of this research has been conducted 
on criminal offenders and psychiatric patients, populations to which the perpetrators of targeted 
school violence may not belong. Although some of the checklists of warning signs and risk 
factors currently used by schools are derived from empirical research on youth violence and 
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aggression, the relationship between these factors and risk of targeted school violence is not yet 
known. 

Fourth, standard psychological tests and other instruments traditionally used in guided 
professional judgments are of questionable utility to school-based targeted violence risk 
assessments. Many psychological tests are designed primarily to assess mental disorders; yet, 
the prevalence, nature, or role of mental disorder among perpetrators of targeted violence is not 
yet known. Nor has research demonstrated any useful relationship between the results of 
standard psychological tests and instruments and the risk of targeted violence in schools (Borum, 
in press). 

Automated Decision-Making 

The two final approaches to assessing risk of targeted violence in schools fall under the 
heading of what we term automated decision-making. They are: (a) actuarial formulas, and (b) 
expert systems and other artificial intelligence / artificial intuition approaches. We review these 
together because both procedures produce a decision (although one that can be framed in more or 
less definitive terms), rather than leaving the decision to the person conducting the assessment. 

Description and threshold of concern . Actuarial tools are equations consisting of 
weighted risk factors that are statistically or mechanically combined to yield a decision about the 
likelihood of a condition or outcome (see e g., Dawes, Faust, & Meehl, 1989). They are created 
based on empirical research on the behavior in question (e.g., violence), and typically are 
standardized for specific populations and subtypes of the behavior in question (e.g., sexual 
assault or domestic abuse). The development of such formulas is optimized when there exists a 

sufficient knowledge base of known variables that contribute to the outcome in question (e.g., 

). / 
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risk factors for a particular type of violence within that population), and where there is a 
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sufficient frequency of occurrence of the event / outcome to permit statistically-derived 
prediction. Where such actuarial equations can be standardized and validated, they have been 
shown to perform as well or better than clinical judgements in a range of decision tasks (Borum, 
in press; Borum et al., 1993; Dawes et al., 1989; Grove, 2000; Grove & Meehl, 1996). In short- 
term assessments of violence risk, however, the formulas have not been significantly more 
accurate than even unstructured clinical judgments (Mossman, 1994). 

Expert systems and artificial intelligence / intuition are defined here as computer-based or 
automated applications of expert knowledge on a particular issue to solve a problem or render a 
decision in an instant case. Through various methods and structures, expertise that has been 
compiled on a particular topic or issue is represented in a computer program through the use of 
algorithms or other computer-based rules (see Beaumont, 1991; Fox, 1996). The computer then 
compares its store of expertise to the facts in the instant case and arrives at a decision or 
outcome, based on the rules in its program and the content of expertise to which the case was 
compared. The advantage of using an automated system is the reduction in any errors introduced 
through human involvement in the decision, such as through biased information collection or 
subjective decision-making (Borum, 1996). The threshold for concern in both actuarial tools and 
expert systems is determined by the formula or system, in theory based upon an empirical or 
experiential knowledge base about the issue. 

Attendant problems . We have three main concerns about the use of automated decision 
making to assess risk of targeted violence in schools. First, appropriate actuarial equations do 
not yet exist to determine risk of targeted violence, particularly school-based targeted violence. 
As we noted earlier, a sufficient knowledge base on the antecedents and risk factors for targeted 
violence in schools has yet to be created, thus precluding the derivation of any meaningful 
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statistical equations to assess the likelihood of its occurrence from known risk factors. More 
importantly, however, the base rate of targeted school violence is too low for any statistically- 
derived equation to attain any reasonable discriminant accuracy. Any equation derived from 
empirically researched risk factors for targeted school violence would never be sufficiently 
sensitive (minimizing the number of false negatives) nor specific (minimizing the number of 
false positives) to reasonably estimate the probability that a given student would engage in 
targeted violence in school (Sewell & Mendelsohn, 2000). 

Second, expert consensus on evaluating risk of targeted violence generally, and targeted 
school violence in particular, has not yet been formed. This area of research is clearly in its 
infancy, with the first known empirical study of violence targeted against public officials and 
public figures just published in 1999 (Fein & Vossekuil, 1999). As previously noted, the 
appropriate empirical research on targeted school violence has yet to be conducted, making the 
utility of existing knowledge on other forms of youth violence as yet unknown. Expert systems 
and artificial intuition programs that claim to compare the student in question with thousands of 
known cases (e g., Morse, 2000; Steinberg, 2000) are not comparing the student to cases of 
targeted school violence because the incidence of such cases is far lower (see Henry, 2000). 

Finally, research on the use of expert systems in other contexts has raised concerns 
regarding the creation of expectations that exceed what expert systems can reasonably 
accomplish (Winegrad & Flores, 1987). To the extent that existing actuarial formulas and expert 
systems are not yet informed by empirical research on targeted violence in schools, they may fail 
to gather information on the student or situation that may be relevant to appraising risk and thus 
produce a flawed assessment. Still other research has documented that users of expert systems 
may rely inappropriately on the decisions produced by a computer (Will, 1991). In one study. 
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users an expert system (both experts and novices for the task in question) reported 
considerable satisfaction with what were in fact flawed decisions the system produced (only one 
participant figured out the decision was fundamentally flawed; Will, 1991). By extension, when 
an expert systems approach is used to determine risk of targeted school violence, there is a risk 
the user may discount their own knowledge of the student in question and rely primarily, if not 
solely, on the computer generated decision instead. 

Threat Assessment Approach 

The common conceptual element in each of these approaches is that they are 
fundamentally inductive (see Turvey, 1999b): they rely on aggregate information about prior 
events to guide inferences about facts in a specific case. In our view, what is needed to evaluate 
the risk of school-based targeted violence posed by students is an approach that is deductive (see 
Turvey, 1999c): one that focuses primarily on the facts of the particular case in question to guide 
inferences (rather than on a series of factors shared by similar perpetrators or other violent 
youth); that examines closely the progression of ideas and planning behaviors over time; and, 
that corroborates information gathered in the case from multiple sources in contact with the 
student. 

Based upon their empirical research ©h assassinations and attacks of public officials and 
public figures, Fein, Vossekuil, and colleagues (1998; 1999; Fein et al., 1995) developed the 
threat assessment approach, a framework for identifying, assessing, and managing persons who 
pose a risk for targeted violence. Threat assessment is guided by several operational principles 
and relies on key questions that this research suggests are important to ask when evaluating the 

risk posed by an individual for acts of targeted violence (Borum et al., 1999; Fein & Vossekuil, 

f 
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1998, 1999; Fein et al., 1995). We believe this approach holds promise for assessing risk of 
targeted violence in schools. 

Guiding principles of the threat assessment approach . Certain guiding principles derived 

from the public official violence research underlie the threat assessment approach. First among 

these is that there is no profile or single “type” of perpetrator of targeted violence. Rather, 

violence is seen as the product of an interaction among the perpetrator, situation, target, and the 

setting. In their study of assassins and near-assassins, Fein and Vossekuil found a wide range of 

ages, both genders, varying educational backgrounds, and other differing demographic features 

(Fein & Vossekuil, 1998, 1999). It is possible that school shooters will show similar 

heterogeneity, as prior research attempting to identify the prototype of a violent youth has been 

wholly unsuccessful. As Herbert Quay noted over a decade ago: “The assumption that all 

delinquents exhibit some common set of psychological characteristics has been the basis for 

most of the early research into the psychological characteristics of delinquents... and 

unfortunately, remains so ...If, in fact, delinquent youth are behaviorally and psychologically 

heterogeneous, the search for single psychological variables that can reliably separate 

delinquents from non-delinquents is not an effective research strategy” (Quay, 1987; p. 118). We 

$ 

extend this comment by suggesting that it is also not a useful clinical assumption or an effective 
assessment strategy. 

The second key guiding principle underlying the threat assessment approach is that there 
is a distinction between making a threat (expressing, to the target or others, an intent to harm the 
target) and posing a threat (engaging in behaviors that further a plan to harm the target). Many 
people who make threats do not pose a serious risk of harm to a target. Conversely, many who 
pose a serious risk of harm will not issue direct threats prior to an attack. For example, no 
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public official / public figure assassin or attacker directly threatened their target prior to the 
attack (Fein & Vossekuil, 1998, 1999). The implication derived from this finding is that, while 
all threats (direct, indirect, conditional, or otherwise) should be taken seriously, they are not the 
most reliable indicator of risk and therefore should not be a necessary condition to initiate an 
inquiry or preliminary evaluation. Indeed, a youth who is committed to mounting an attack may 
be less inclined to threaten a potential target directly, particularly if he or she does not want to be 
stopped. The youth may, however, discuss ideas of harm among friends and peers. 

The third assumption underlying the approach is that targeted violence is not random or 
spontaneous; it does not occur because someone “just snapped.” The research on public official 
/ public figure violence indicates that this type of targeted violence is the result of an 
understandable, and an often discernible, pattern of thinking and behavior (Borum et al., 1999; 
Fein & Vossekuil, 1998; Fein et al., 1995). What this finding suggests is that many incidents of 
targeted violence may be preventable. Conceptually, this principle is very important since 
assessing risk for events that are considered to be random would seem to be a contradiction. If, 
however, they are viewed as the result of a behavioral process, then a fact-based assessment 
makes sense. 

What constitutes threat assessment . 1*he threat assessment approach is a set of 
operational activities that combine the use of an investigative process and information-gathering 
strategies with target-violence relevant questions (see Borum et al., 1999 for a detailed 
description of the threat assessment approach; see also IACP, 1999; p. 67). These activities are 
designed to identify, assess, and manage individuals who pose a risk of violence to an identified, 
or identifiable, target. 




23 



School Threat Assessment 23 



A threat assessment may be initiated by any communication or behavior of concern. 

Threats are not a necessary threshold for concern; however the threat assessment approach also 
dictates that no threat should be ignored. The process of gathering information about the 
individual includes an investigative emphasis on corroboration of facts to establish their veracity 
(in contrast with the typical clinical emphasis on the patient’s story or their perception of events). 
The focus of the inquiry is on the individual’s behavior in the instant case, and what the 
progression of their behaviors may suggest (i.e., movement from development of an idea to 
implementation of a plan). The threshold for concern is evidence that suggests the individual 
may be on a pathway toward violent action. The threshold is deliberately set low enough to 
facilitate early intervention, as the emphasis of this approach is on prevention and the 
development of effective case management strategies. 

The threat assessment approach asks the person conducting the inquiry to gather 
information, and answer key questions about the instant case, to determine whether there is 
evidence to suggest movement toward violent action. The questions focus on motivation for the 
behavior that brought the person being evaluated to official attention; communication about ideas 
and intentions; unusual interest in targeted violence; evidence of attack-related behaviors and 
planning; mental condition; level of cognitive sophistication or organization to formulate and 
execute an attack plan; recent losses (including losses of status); consistency between 
communications and behaviors; concern by others about the individual s potential for harm, and, 
factors in the individual’s life and/or environment or situation that might increase or decrease the 
likelihood of attack. 

Taken together, the information learned from these questions— as gathered from the 

i;: 

student and from corroborating sources (family members, friends, teachers, classmates, school 
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and mental health records, etc.)~should provide evidence to answer the question of whether the 
student is moving on a path toward violent action. The answer to the last issue in particular can 
inform the development of a risk management plan. For example, school officials could decide 
to take active steps to minimize factors that could put the student at greater risk for an attack, 
such as through referral to appropriate services. Or they could opt instead to monitor the student 
(perhaps with assistance from family and others close to the student) for changes in factors that 
could increase the student’s targeted violence risk. 

Conclusion 

When considering how best to prevent (rather than optimally predict) targeted violence in 
circumstances where a student has come to official attention because of threatening or 
concerning behavior, traditional inductive approaches are unlikely to be helpful. The use of 
profiles is ineffective and inefficient, carries with it a considerable risk of false positives (most 
youth who fit the profile are not a targeted violence risk), has a potential for bias, and has been 
sharply criticized for its potential to stigmatize students and deprive them of civil liberties. The 
use of guided professional judgment-while highly appropriate and effective for evaluating risk of 
more general forms of violence and aggression-is currently inappropriate for evaluating risk of 
targeted school violence. The knowledge base of empirically-researched risk factors for targeted 
school violence has yet to be developed, nor has any relationship been established between 
general youth violence risk factors or standard psychological tests/instruments and the 
occurrence of targeted violence. And finally, because targeted school violence is such an 
infrequent event, it is not amenable to statistical prediction by actuarial tools. Nor is it amenable 
to evaluation by expert systems or artificial intuition programs because expert consensus on this 
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topic has not yet been reached and the validity (i.e., accuracy) of the programs or their decision 
rules has not been established. 

We suggest that a deductive, fact-based approach is needed to investigate and assess the 
risk for targeted violence in schools. The threat assessment approach developed by the U S. 
Secret Service represents a good first step toward identifying and assessing risk posed by 
students for targeted violence in schools. We have promoted this approach as one that is suitable 
for use right now by mental health professionals, school administrators, law enforcement 
professionals, and others who have responsibilities for maintaining school safety. We believe, 
however, that what is most needed for effective prevention of planned school-based attacks is 
empirical research on incidents of targeted school violence. Many of the shortcomings that we 
highlighted about current assessment approaches for targeted school violence center around the 
lack of empirical research on targeted violence perpetrated by students at school. We recognize 
that although the threat assessment approach is based upon empirical research on targeted 
violence, it too lacks the benefit of comprehensive empirical knowledge on targeted violence in 
schools. The most effective approach for understanding and preventing planned school-based 
attacks will be the one that is informed by empirically-derived knowledge about the antecedents, 
motives, idea development, communications, 'and planning behaviors of all known perpetrators 
of targeted school violence. We see these as the most critical unanswered questions that school 
and law enforcement professionals currently face in attempting to prevent targeted violence in 
schools. 
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Endnotes 

1 . This definition of targeted violence is intentionally vague. It is not intended as an operational 
research definition, but is proposed as conceptual heuristic to stimulate thinking about the 
differences between the school-based attacks and the more prevalent, less publicized violence 
that commonly occurs in schools and among youth. 

2. It should be noted, however, that the U.S. Secret Service is in the process of conducting the 
first empirical analysis of all known incidents of targeted school violence in the U.S., from 
1974 through 2000. Dissemination of findings is anticipated to begin in Fall of 2000. 

3. While much discussion in the violence risk assessment literature has focused on the accuracy 
of predictions (Borum, 1996; Monahan, 1981; Mossman, 1994; Otto, 1992), we see an 
important distinction between predicting violence and preventing it. The central difference 
lies in the outcomes implied by each term. With the frame of “violence prediction” or even 
“violence risk assessment,” the implicit outcome is maximizing the accuracy of the 
assessor’s predictions-to be able to gauge accurately who is more likely to be violent, and the 
circumstances under which the probability is greatest (Sewell & Mendelsohn, 2000). With a 
frame of violence prevention, however, the outcome emphasis shifts from optimizing 
predictive accuracy to effecting appropriate interventions. By emphasizing prevention as the 
outcome, the need to provide necessary services takes precedence over the need to be “right” 
about whether a given child will in fact become violent. More importantly, we would argue 
that the need to intervene permits school officials and others to consider options that are less 
punitive (e g., counseling, establishing a friendship with the child; finding a mentor; etc.) 
than those available when the emphasis is placed on the child’s danger to others. 
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4. A 1998 Wall Street Joumal/NBC News poll showed that 70% of respondents believed a 
school shooting could happen in their community (“Washington Wire,” April 24, 1998). 

Also, in a critique of television coverage of these infrequent school shootings John Stossel 
(1999) reported that the three major television networks, ABC, CBS, and NBC, aired a total 
of 296 stories on the shooting at Columbine High School in Littleton, CO. Stossel noted that, 
in contrast, lightning accounts for more deaths overall, and bathtub accidents account for 
more deaths of children, than do school shootings (Stossel, 1999, Tebo, 2000), yet, they 
receive comparatively little media coverage. 

5. The Centers for Disease Control and Prevention and the US. Department of Education 
define a school-associated violent death as any homicide or suicide that occurred (a) on the 
campus of a functioning elementary or secondary school (in the U.S.), (b) while the victim 
was on the way to or from regular sessions at school; or (c) while attending or travelling 
to/from an official school event (e.g., a football game or school dance; Kachur et al., 1996). 

6. It is not currently known how many schools use which type of assessment. No data yet exist 
that describe the prevalence of any of these three approaches (or others) schools may 
currently use, nor of their effectiveness - perceived or actual. Researchers at the Louis de la 
Parte Florida Mental Health Institute, University of South Florida, the U S. Secret Service 
National Threat Assessment Center, and the Department of Education’s Safe and Drug Free 
Schools Program are currently surveying school administrators, school-based law 
enforcement personnel, and others law enforcement investigators on this issue. The purpose 
of these surveys is to describe the “landscape” of current assessment approaches used to 
evaluate school-based targeted violence risk, gauge the perceived effectiveness of those 
approaches, and identify perceived barriers to improved evaluation effectiveness. 
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7. The threshold for concern is not specified with sufficient clarity in the school shooter 
profiles, nor in the warning sign checklists. In a few places, the instruction given indicates 
that the greater number of characteristics present, the greater should be the evaluator’s 
concern (e g., Dwyer et al., 1998). Others in the field of violence risk assessment have 
argued that this may not be an appropriate threshold (e g., Borum et al., 1993). 

8. Although we postpone discussion of warning sign checklists until the following section (on 
guided professional judgement), we recognize that such lists belong equally in a discussion 
of concerns about profiling (Sewell & Mendelsohn, 2000). We note that the Department of 
Education warning sign checklist explicitly states that their list does not constitute a profile 
and cautions that the list was never intended to serve as a predictor of violent behavior; 
rather, it was intended to provide educators and others working with youth with a better idea 
of which students needed help and assistance (see e.g., Dwyer et al., 1998). However, 
despite such cautions, others have argued that in school settings warning sign checklists are 
often applied as profiles and their explicit cautions disregarded (Sewell & Mendelsohn, 
2000). We believe the concerns about profiling for identifying and assessing risk of targeted 
school violence also apply to the use of warning sign checklists for the same purposes. 

9. With respect to the utility of current school shooter profiles, one aspect of the FBI school 
shooter profile in particular bears scrutiny: The FBI’s school shooter profile includes the 
characteristic that prior school shooters showed no remorse after the shooting (Band & 
Harpold, 1999). We question the utility of including a behavior that can only be detected 
after an incident (no remorse over the shooting) in a profile intended to be used to identify 
children before a critical incident occurs (see also Sewell & Mendelsohn, 2000 for a similar 
discussion of violent behaviors included in profiles and checklists for violence risk). In 
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interviews conducted by the Secret Service with several recent school shooters, many of the 
shooters have shown considerable remorse over their actions (see 60 Minutes II, 2000). 

10. The Department of Education emphasizes that the warning sign list included in their 
publication. Early Warning. Timely Response , was never intended to serve as a predictor of 
youth violence but rather to provide a better idea of which students may need assistance. 

1 1 . It is important to distinguish between “risk factor” and “warning sign. These terms have 
been used interchangeably in the broader public debate about profiles, warning signs, and 
school violence prevention. We emphasize that the concept of a risk factor is based upon an 
empirically-established relationship with a particular outcome. Last (1988) defines risk 
factor as an attribute or exposure that is associated with an increased probability of a 
specified outcome, such as the occurrence of a disease (although not necessarily a causal 
factor). In contrast, a warning sign does not necessarily imply an empirical relationship with 
a particular outcome, but rather is used as a rule of thumb or heuristic to suggest the presence 
or impending existence of that outcome (see American Heritage Dictionary, 1996). 

12. As a point of clarification, the distinction between guided professional judgment and 
actuarial assessment is the clinician’s involvement in any decision or judgement. Any 
assessment approach where the evaluation is made by means other than a strict statistical or 
formulaic combination of variables is considered to be clinical assessment (Borum, in press). 
Please refer to the section on automated decision making for a more in-depth discussion of 
the use of actuarial formulas in violence risk assessment decisions. 

13. Interestingly, this concern about over-reliance on base rates in targeted violence risk 
assessments runs counter to the more common concern that general violence risk assessments 
do not take base rates into account sufficiently (e.g., Borum et al., 1993). 
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